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ABSTRACT 

Statewide competency testing as a high school 
graduation requirement represents a threat to those who encourage 
broad notions of reading and writing, even when a number of testing 
innovations in reading and writing mean a step forward in education. 
In Alberta, Canada, the English exam comes in a non-academic and an 
academic version, and contributes 50% of the student's final grade. 
The writing component of the test requires an expository essay from 
the academic students and a piece of "functional" writing from the 
non-academic students. While the selections for the reading component 
can be considered progressive (e.g., works by Joan Didion, Margaret 
Atwood), the format is multiple choice, which eliminates the multiple 
meanings encouraged in reading works of any subtlety. English 
teachers surveyed indicated that their classroom programs had been 
affected in ways that contravene the intentions and spirit of the 
examinations., including: (1) changes in regular tests to correspond 
more closely to the comprehensive exam format; (2) increased use of 
multiple-choice testing in the literature program; and (3) new 
importance among administrators as tests became contests among 
schools. ' Such curricular compulsion of exam preparation remains a 
major source of distortion that can render even the most promising 
approach a rote and remote exercise. (Examples of writing assignments 
are included. (NKA) 
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WHEN TESTS DAPE TO BE PPOGFESSIVE: CONTPADICTIONS IN THE CLASSPOOM 



The Context; Political Demand and Pedagogical Promise 

State wide-testing, whether as a competency measure or a 
criterion-referenced standard, has swept the continent taking hold in 3 6 
states and six provinces (Blair^ 1985, p. 18). The threat of the tests to 
those who would encourage broad notions of reading and writing has been 
represented as reducing classroom work to irrelevance (Cooper, 1981/ p. 5), 
displacing the central place of literatxire (Myers, 1981, p. 166) and promoting 
a "Issser definition o'" literacy" (Probst, 1982, p. 22). This paper explores 
that threat in a promising guise; it presents a progressive instance of the 
tests examining both the nature of their construction and their impact on the 
English classroom. The province of Alberta, in reinstituting diploma 
examinations for high school graduation after an absence of a dozen years, has 
brought to its testing program a nxamber of innovations in reading and writing 
which have meant a step forward in education, but a step that still tends to 
falter in the examination f rmat. 

To portray this dilemma, the paper is divided into two parts, into a 
close reading of the examinations and a report on the perceptions of grade 
twelve English teachers on the effect which the examination had have in the 
two years of their implementation. With the Alberta Diploma Examinations, we 
have a most interesting experiment, a strong deliberate effort not to go back 
to the basics, but to marry political necessity for greater accountability to 
the prcgrossive gains in English education over the last two decades, as 
politician might seem to embrace teacher in some sort of ideal state north of 
the Montana border. The Alberta exams can best be described as an instance of 
precious new wine in rather old bottles. The question we wou2d pose is 
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WHEN TESTS DARE TO BE PROGRESSIVE / 2 

whether the final kick comes from the wine or its bottling, or to reverse the 
formulation by drawing on Marshall McLuhan, whether the mediiam is not 
inevitably the message. It should be noted that one of the authors, Allen 
Bobie, has been a participant-observer in this study, that is, he has prepared 
grade twelve students for the exams since their re-introduction and 
participated in the marking of the diploma examinations. As will be seen, his 
experiences serve as a touchstone in this analysis. 

The Tests; Fresh Wines in Dusty Bottles 

The Alberta Diploma Examinations can be written in six subjects at the 
end of grade twelve. The exam in English comes in an academic and 
non-academic version of the English exam, the only subject which must be 
written for graduation. The examination contributes fifty percent of the 
students* final grade, with course work counting for the other fifty. In 
describing the rationale for the examinations, the provincial department of 
education openly acknowledges the limitations of paper and pencil testing. 
The exams are only meant to "complement" the curriculum and the teacher's 
program? they are closely based on the curriculum specifications already in 
place in the province. Ultimately, the exams are meant to "reflect the 
wholeness of the Language Arts program," as one provincial docvunent puts it 
(Alberta Education, 1983, p. 1). So far, we might conclude, so progressive, 
and yet there is more to this promising new vintage. 

The exams are divided into two sittings. Part A; Written Response , as 
the first part is termed, reflects the influence of both the writing process 
and James Britton's (1972) construct which moves from the expressive to the 
transactional. However, this progressivism is sustained only on the academic 
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version of the exaii^ination. At the acs^'^:'^' irr Jevel, in response to two brief 
pieces of literature the student first in a personal vein, then do a 

brief critical piece, finally arrivirto a\. . could be termed a transactional 
piece of expository prose on a literary x:he. :;ee Figure 1). In each case, 
outlining and rough work are encouraged and pr*-- ided for. Throughout the 
Written Response, the student continues to q jv^lop a coimnon theme in response 
to the two works. 



Place Figure 1 here 



The possibility of a worm in this apple of an examination becomes 
apparent in the preparation which teachers have begun to provide their 
students for this response- Bailed on his experience as a marker of the exams, 
Allen Bobie coaches his grade twelve students for the personal response on the 
exam by suggesting that they would do well to keep their own first responses 
in check, in favor of what they would project an adua t/teacher might welcome 
as a personal response from a responsible student. This sort of coaching to 
second-guerpS adult expectations has begun to dominate the college admissions 
essay (Winerip, 1987). Recognizing that this sort of bottling taints the new 
wine, Bobie points out that this form of "personal response" is still that 
much more sophisticated, for all of its projection and contortion. 

The examination for the non-academic students lacks the Brittonesque 
developmental structure; it begins with a personal response to a reading, but 
then moves to "functional" writing, and, in what might be taken as a 
progressive gesture, the final question is on visual literacy. It asks the 
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students to interpret a cartoon, often enough from The New Yorker and once 
from Jules Feiffer's work from the Village Voice > This represents more than a 
failure to share the benefits of Britton's boon to writing with all students. 
It implies a future in the langxiage of looking for work and mis-reading 
sophisticated funnies- -and once work has been fovind? While for the academic 
students, the exams seem to foresee the nurturance of extended responses, 
personal and public^ to ti:e themes of responsibility, triumph and 
self- awareness, to draw on themes from three sets of exams. As Judith Chinn 
points out in her examination of testing in California, "Even students who go 
to Harvard need to apply for jobs" (19&3, p. 15), And the reciprocal point, 
that non-academic students may need to argue a critical response to a 
situation, might also be thought to hold. 

The second sitting for the students is entitled Part B; Reading ( Multi ple 
Choice^. Though on first glance it may not seem so, the progressive element 
is once again present if not triimiphing. Here it emerges in the reading 
selections, in the excerpting of Joan Didion, Margaret Atwood, Lillian Hellman 
and Ellen Gilchrist. By the second year of the exams the selection was more 
progressive in literary style and gender balance than anything else the 
students were likely to receive in their dated school anthologies. 
Shakespeare was present at the academic level exams without fail, and yet that 
was essentially it for pre-twentieth century literature in the certification 
of reading at the advanced high school level. It may be no way to introduce 
students to what is exciting and nev in literature, and it may be no way to 
treat the history of literature, but it does reflect a progressive urge, and 
teachers, if no one else, might be moved by it to ^jonsider fresh voices for 
the classroom. 
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That much said, the regressive faults of the multiple choice tf!st for 
literature need little elaboration: the multiple choice question demands the 
rapid elimination of the multiple meanings which we would often encourage in 
reading work of any subtlety, it runs contrary to. much that the recent report 
Becoming a Nation of Peaders holds to define the constructive, self^ directed, 
motivated nature of reading 'Anderson, Heioert, Scotu, Wilkinson, 1985). In 
fact, "multiple choice test" would seem a misnomer — the choice is far too 
singular, with Shakespeare, we found that the format favors decoding, on 
getting what the playwright really meant by this word or that phrase, if 
only he had had it in him to tell us as much. This, then, is the students 
final lesson on the nature of literature. 

While the marking of the misnamed multiple choice examinations is a 
matter of machines, the assessment of the students' written response 
constitutes what might be construed as an expensive state-wide experiment in 
program implementation. Teachers from across the province are invited to a 
week-long junket in the capital city to team-mark the written responses at an 
hourly rate. Each paper is marked by three teachers on a number of five-point 
scales including thought, organization, choice, and convention with the major 
assignment. The efforts at quality control througl. training sessions and 
daily marking statistics show a progressive tendency reflecting what we know, 
for example, about holistic marking. It also created for the markers an 
intense immersion course in the nature of mass evaluation, as the curriculum 
initiative in the province moves from the Curriculum Development office to the 
Evaluation branch. The diploma exams in English represent a mix of 
progressive and regressive aspects. The final test in assessing their impact 
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and the balancing of this mixtiixe .lies in cheir impact on the English program, 
on which aspects have been felt in the classroom. 

The Contest; Turing the C\irricul\nn To the Tests 

In seeking the effect of the examinations on the teaching of English, we 
decided to work with a sample of those teachers who had marked in Edmonton. 
The province's plan to rotate the marking among teachers means that eventually 
the most^of the high school English teachers will pass through this process; 
we also knew that in judging the impact of the examinations, then only two 
years in place, these markers would be the first to show signs of its 
influence in their curricular thinking. A brief survey was designed to tap 
these teachers perceptions of the examination's impact on their program and 
was sent to seventy-five teachers with recent experience marking; we had a 
return rate of forty-five or sixty percent. As a result, we found a number of 
areas in which the teachers felt their program had been affected, affected in 
ways that, unfortunately, contravene the intentions and spirit of the 
examinations. 

The first aspect of influence is in the teachers design of writing tasks 
in the classroom. The question put was "How often do yoa create essay topics 
for students using the diploma exam format?" (see Table 1). Though the 
responses to the survey lack a fixed numerical value, it was designed to 
capture the teachers' sense of how "seldom," "o<;casional" or "frequent" the 
exams encroach on the program. In this case, close to half of the respondents 
find that with some frequency the exam shapes the topic of their classroom 
essay. This has the somewhat ironic effect of introducing the promising 
aspects of the exam into the classroom, but in the form of exam preparation 
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rather than as part of a more widely applicable writing process, while the 
recent national reports^ such as The Writing Peport Card (NAEPr 1986), call 
for for this extended , critical writing, finding all too little of it 
currently in the schools, it seems clear that if it becomes training in a 
single form, the transfer value of the exercise will be considerably hampered. 
It risks becoming a dead-end skill. The ol^^r indication of a distortion 
through exam preparation was the frequency of timed responses (20-25 minutes) 
in such areas as personal response and literary analysis, with a few teachers 
reporting that old examinations are used to set the direction of expository 
writing in the class. 



Place Table 1 here 



The other side of this writing after the examinations is in che teachers' 
evaluation of the students' work during the term. The exam format of 
categorical assessment on one-to-five scales has achieved degree of frequency 
among half the teachers (Table 1). Again tc be noted that while the four 
divisions (total impression, thought and detail; organization, matters of 
choice, matters of conventions) in the scoring are the same — some schools had 
rubber stamps made with the four categories — a good deal of the holistic 
advantage has been lost. 

The format of the diploma examination, with both written response and 
multiple choice sections, had also begun to pervade evaluation procedures 
across the senior high grades of grades ten to twelve; sixty-seven percent of 
the respondents responded yes to the query "has your school changed the format 



EKLC 



9 



WHEN TESTS DAPE TO BE PPOGPESSIVE / 8 

of grade ten and eleven exams to correspond more close?.y to the diploma 
exams?" Mien Bobie, in whose school this practice has been pursued over the 
past two years ^ has found that the element of preparation for the diploma 
examinations is the understood reason for following this format in the other 
grades, rather than any sense oi its improved pedagogical status as an 
assessment instrument. 

In the area of reading, the greatest sense of influence emerged in the 
increased use of multiple-choice testing as part of the literature program. 
l!he survey put the question to the teachers in terms of the amount of use 
prior to the implementation of the examinations in 1984 and the amount of use 
currently, in 1986 (Table 1). Not surprisingly, the rise is dramatic. To 
give an instance of the impact of this on the program in terms of classroom 
time, Allen Bobie spends at least six periods or close to five percent of his 
teann in preparing students with multiple-choice testing; it is for him the 
clearest case of pursuing a questionable technique in the teaching of 
literature in order to prepare his anxious students for the diploma 
examinations. At such a point, the notion of the exams complementing the 
current cxarriculum begins to crumble. 

Other instances of this movement from complement to dominance emerged in 
terms of the teachers' curriculum priorities. Through the use of a checklist, 
teachers assessed changes in the emphasis they gave to twelve particular , 
language skills. A greater emphasis was noted by the teachers in the 
importance of reading and writing skills, along with vocabulary development, 
which is certainly to the exams' credit, while those suffering in this shift 
were creative writing, speaking and listening skills, and viewing skills. In 
particular, th^> library research essay was singled out as something that 
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loeing its place in the grade twelve ciirriculum; the teachers whom we spoke to 
felt that it did not does not adequately address the focused type of response 
which the diploma exam demanded. Yet, of all the work done in that final year 
of high school among academic students, the library research paper comes 
closest to an experience in university-prep, in taking a try at the dominant 
university genre. 

As a final measure of the examination's influence on the classroom, the 
survey explored perceptions of the administrator's concern with performance on 
the examinations. In response to the query "What importance is placed by 
superiors on your students' exam performance in relation to the provincial 
average?" we found the majority felt that this meeting the average had taken 
on at least some importance among administrators as tests become contests 
among schools and boards (Table 1). it has become the one easy measure of a 
certain kind of productivity, and toward this end of teacher evaluation, Allen 
Bobie has been asked by his principal to complete a class summary of student 
results for each of his classes, though the province issues only school-wide 
results. 

Conclusion 

The Diploma Examinations of Alberta represent a marriage of political 
accountability and pedagogical advancement. They serve to notify the teachers 
of the reality of the times, as well as introduce them to such advancements as 
writing from process and to recent openings in the canon of readings. The 
urgei.t question is whether the examinations can serve both ends. Progressive 
practices do not lend themselves easily to province-wide examina::ions . 
Clearly, some of this resulting distortion can be remedied through an ongoing 
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and sensitive revision of the exams. Yet it also seems apparent from our 
modest survey that the major source of distortion remains the curricular 
compulsion of exam preparation, an urge which can render even the most 
promising approach into a rote and remote exercise. 

This damaging urge is a function of accountability; it results naturally 
enough from a high profile, province-wide testing program. The pressure to 
prepare is felt by everyone in the system from superintendent to student. 
Certainly, it is no less visible in the anxious students whom the grade twelve 
English teacher now faces; the students know better than anyone how the tests 
count. And this becomes the ciirricular imperative above all other concerns in 
the students' final year of high school language and literature. The 
immediate challenge to this province and to other jurisdictions struggling 
with this political pressure to test on a massive scale is to stand aware of 
the imminent dangers of such a program for even the best intentioned plans. 
The tendency is for the examinations to simply overwhelm the curriculum, 
rather than serve to guide and instruct. We are not all certain that the 
distortion of mass testing is completely rectifiable; we can only advise that 
an openness to both vigilance and refinement will be necessary to see this 
marriage of politics and pedagogy harmoniously realized. 
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FIGURE 1 

The Alberta Diploma Examinations in English (January 1986) 
PART A: WRITTEN RESPONSE (Academic) 
1. Personal Response Assignment 

In the story "Harrison Bergeron" Vonnegut describes a society in which 
people have been made equal "every which way." If you lived in such a 
society, how would you feel and what would you do? 

3. Major Assignment 

Societies are shaped by such factors as geography, history, economics, 
politics, the family and religion. Authors of such works as "Looking at 
Models in the Sears Catalogue" and "Harrison Bergeron" suggest that people 
are strongly influenced by the prevailing conditions, attitudes and values 
of the society in which they live, whether they are aware of the influences 
or not. Compare two selections in which the authors show how people are 
affected by or respond to the prevailing conditions, attitudes or values of 
their society. 
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PART A: WRITTEN RESPONSE ( Non- academic ) 

2. Fvinctional Writing 

Plan NOW For Your Future Occupation! 

CAREERS DAY 

June 21, 1986 
Nalwen Composite High School 

Don't Miss this Important Event 

In the space provided, write a letter inviting your speaker to Nalwen 
Composite High School . . . 

3. Response to Visual Information 

[The page contains a New Yorker cartoon which has a row of six identical 
and partially constructed houses on top of which sit carpenters who have all 
at the same moment hammered their fingers, shades of "little boxes of 
ticky-tacky'] 

Describe how you would feel if you were one of the characters. . . 
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Table 1 

Pesponses to Survey on the Influence of Diploma Examinations 



Abbreviated Questions (N=45) A. B. C. D. 



1. 


Follow 


exam format for essay topics 


5% 


9 


38 


36 


12 


2. 


Follow 


exam format for marking essays 


6% 


10 


31 


28 


24 


3. 


Use of 


multiple choice a. Pre-exam 


14% 


22 


48 


12 


4 






b. Currently 


4% 


14 


36 


41 


5 


4. 


Stress 


placed by superiors on results 


4% 


9 


43 


30 


14 



Note. Pesponse Categories: A. Very Seldom; B. Seldom? C. Occasionally; D. 
Frequently; E. Very Frequently. 
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